Skip to content

feat(agent-stream): thinking and tool streaming#5671

Merged
icecrasher321 merged 33 commits into
stagingfrom
agent-stream-events
Jul 24, 2026
Merged

feat(agent-stream): thinking and tool streaming#5671
icecrasher321 merged 33 commits into
stagingfrom
agent-stream-events

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds agent-events-v1 streaming for live thinking + tool lifecycle across Anthropic, Gemini, Bedrock, DeepSeek/Groq (openai-compat), with dual-gated deployed-chat SSE and canvas console chrome.
  • Wires DeepSeek/Groq/OpenAI reasoning request knobs, tool-loop reasoning round-trip, timeout/user abort semantics, and /api/providers byte projection for object streams.
  • Ships ChatGPT-like thinking chrome (fixed scroll pane, follow-to-bottom, auto-collapse when the answer starts) plus includeThinking chat policy/migration/docs.

Test plan

  • Deployed chat with Include thinking on + Anthropic/Gemini reasoning model: thinking streams in fixed pane, collapses when answer starts, tools show as chips
  • DeepSeek chat/reasoner and Groq gpt-oss with thinking/effort: thinking events appear; forced tools still complete
  • OpenAI reasoning model with effort auto: summaries still stream
  • Canvas run with thinking model: console shows thinking/tools; Stop soft-completes without a scary failed agent block; timeout still fails the block
  • Legacy client without protocol header: byte-identical SSE (no thinking frames)
  • bun run test / vitest for agent-stream, providers, streaming, chat hook suites

Made with Cursor

… and canvas

Ship the agent-events-v1 protocol with provider tool loops, dual-gated chat thinking, DeepSeek/Groq/OpenAI reasoning wiring, and ChatGPT-like thinking chrome.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 24, 2026 2:16am

Request Review

Comment thread apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts
Biome was failing CI on migrations/meta/0261_snapshot.json. Also settle
assistant streaming/tool flags when SSE ends without a terminal frame,
without clobbering Stop's finalized content.

Co-authored-by: Cursor <cursoragent@cursor.com>
Auto-format the sim package for CI lint:check, and repair the Anthropic
streaming tool-loop payload after an unsafe delete-to-undefined rewrite.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread apps/sim/providers/stream-pump.ts
…journal test

Treat AbortError from reader.cancel as a cancelled pump result so soft-complete
retains answerText. Point the workspace storage migration journal assertion at
0261_chat_include_thinking.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts
Ignore terminal SSE error frames after the user aborts so
"Client cancelled request" cannot overwrite "Response stopped by user".

Co-authored-by: Cursor <cursoragent@cursor.com>
… scroll

Add left-to-right shimmer on live thinking label/body, keep scroll working by
shimmering an inner node, and follow the answer only while near the bottom.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread apps/sim/providers/stream-pump.ts
Comment thread apps/sim/executor/execution/block-executor.ts
…ts only

Abort the agent stream pump when the projected HTTP body is cancelled so
provider work does not continue after disconnect. Limit AbortError soft-success
to Agent blocks so Function/HTTP cancels still fail in logs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Paused chat runs with Include thinking enabled were dropping the flag when
serializing the pause snapshot, so resume always rebuilt streams without
thinking/tool SSE frames.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread apps/sim/executor/execution/block-executor.ts
Persist pump answerText onto the streaming execution before throwing on
timeout, and carry that partial content into the failed block output so
logs match what the client already saw.

Co-authored-by: Cursor <cursoragent@cursor.com>
Match thinking UX: open while tools run, collapse when finished, and keep
the panel open only if the user manually reopens it.

Co-authored-by: Cursor <cursoragent@cursor.com>
Clear agentStreamActive and settle running tool chips when blocks error,
timeouts cancel runs, or execution ends without stream:done so the output
panel does not stay on live Thinking/Using tools chrome.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts
…helper, annotate remaining legacy casts

Audit of all 26 providers for the agent-events feature confirmed every
streaming execution declares agent-events-v1 and every adapter emits
AgentStreamEvent objects. Cleanup from the audit: the unconsumed legacy
createOpenAICompatibleStream byte helper is deleted, and the remaining
streamResponse-as-any casts (xai, nvidia, kimi, meta, zai, sakana) are
annotated typed casts matching the groq/deepseek fix.
Comment thread apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts
…nd protocol

The live tool loops buffered all answer text per model turn (classification
of intermediate vs final is only known at turn end), so gated surfaces saw
thinking stream, then dead air with the thinking chrome stuck open, then the
whole answer at once.

Loops now emit text deltas live as `turn: 'pending'` plus a `turn_end`
event per turn. The pump buffers pending text and projects it to the byte
path (answerText/logs/memory/legacy clients) only on a final turn_end, so
all settled semantics are unchanged. Gated surfaces render the pending text
as it streams and reconcile with a reset when a turn resolves to tools:

- public chat: live `chunk` frames from the sink + dual-gated `chunk_reset`;
  byte-path frame emission is suppressed to avoid duplicates (kept for
  response-format transformed streams via clientStreamTransformed)
- canvas: forwarder emits live `stream:chunk` + `stream:chunk_reset`; the
  execute route and HITL resume readers stop re-emitting byte chunks; panel
  chat tracks per-block segments and replaces content on flush
- chat client: per-block text segments, chunk_reset handling, and thinking
  chrome now settles on tool start as well as first answer chunk
Comment thread apps/sim/app/(interfaces)/chat/[identifier]/chat.tsx
…ing and reset reconciliation

Three-reviewer pass over the branch, findings validated against staging:

- agent-handler forwards agentEvents to executeProviderRequest — the flag was
  computed but dropped in the field-by-field copy, so provider-side thinking
  requests (OpenAI summaries, Gemini includeThoughts, Anthropic summarized
  display) never activated on opted-in runs
- openai: restore summary:'auto' alongside explicit reasoning effort — staging
  always paired them; gating summary purely on agentEvents changed legacy
  payloads
- gemini: Gemini 2 + tools + responseFormat falls back to the silent path;
  the live loop never applied the deferred responseSchema for AUTO tools
- openai-compat loop: malformed tool-argument JSON fails the call instead of
  executing with defaulted {} args (staging parsed inside the execution try)
- openai-compat parser: a vendor id arriving after a synthesized start no
  longer renames the call (start/end ids stayed consistent)
- stream-pump: abort closes the byte projection so a drain blocked on
  backpressure cannot deadlock teardown
- chunk_reset removes the block from the client text order (deployed chat +
  panel chat) so a reset block re-registers at arrival position — fixes
  separator/order corruption when parallel blocks stream around a reset
- resume route echoes the negotiated X-Sim-Stream-Protocol response header
  (parity with the chat route); docs: [DONE] wire shape + final-vs-error
  terminal semantics corrected
…-age gate

CI's bun install --frozen-lockfile blocks 0.114.0 (published 2026-07-23,
younger than the 7-day supply-chain gate). The pin is exact and was vetted
for the agent-events streaming work; following the existing bunfig pattern,
the exclusion ages out on 2026-07-30 and should be dropped then.
…e strict boundary audit

The audit only recognizes the annotation on the line directly above the cast;
two annotations had drifted behind intervening code lines (groq stream params,
deepseek loop messages) and the OpenAI reasoning-summary widening cast was
never annotated. No behavior change.
@icecrasher321 icecrasher321 changed the title feat(agent-stream): thinking and tool streaming for chat and canvas feat(agent-stream): thinking and tool streaming Jul 24, 2026
…ilure

A failed run can still terminate with a `final` frame carrying success: false;
running chips previously settled green regardless of the outcome.
@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile-apps

@icecrasher321

Copy link
Copy Markdown
Collaborator

@cursor review

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds an opt-in agent event streaming protocol and corresponding provider, executor, deployment, and UI support.

  • Streams provider-exposed thinking, answer resets, and tool lifecycle events across supported providers.
  • Adds dual-gated deployed-chat exposure through the includeThinking policy and protocol header.
  • Adds chat and canvas UI chrome for live thinking and tool status.
  • Extends persistence, execution metadata, documentation, migrations, and tests for the new behavior.

Confidence Score: 5/5

The pull request appears safe to merge with no eligible blocking follow-up failures identified.

No blocking failure remains within the scope of the displayed prior-review threads.

Important Files Changed

Filename Overview
apps/sim/lib/workflows/streaming/agent-stream-protocol.ts Defines the opt-in protocol and deployment-policy gate controlling agent event exposure.
apps/sim/providers/stream-pump.ts Centralizes provider event consumption, answer projection, and stream lifecycle handling.
apps/sim/executor/execution/block-executor.ts Integrates streaming agent executions with executor completion, cancellation, and timeout behavior.
apps/sim/lib/workflows/streaming/streaming.ts Projects internal agent events onto workflow and deployed-chat streaming surfaces.
apps/sim/app/(interfaces)/chat/hooks/use-chat-streaming.ts Consumes the new chat event frames and updates assistant thinking, answer, and tool state.
apps/sim/stores/terminal/console/store.ts Stores and settles canvas terminal thinking and tool lifecycle state.
packages/db/migrations/0266_spotty_alex_power.sql Adds the default-off persisted policy controlling deployed-chat thinking exposure.

Sequence Diagram

sequenceDiagram
  participant Client
  participant ChatAPI
  participant Executor
  participant Provider
  participant UI
  Client->>ChatAPI: Request with agent-events-v1 header
  ChatAPI->>ChatAPI: Check includeThinking policy
  ChatAPI->>Executor: Execute with agent event sink
  Executor->>Provider: Start streamed agent/tool loop
  Provider-->>Executor: Thinking, text, and tool events
  Executor-->>ChatAPI: Forward gated stream events
  ChatAPI-->>Client: SSE agent-events-v1 frames
  Client->>UI: Update thinking, answer, and tool chrome
  Provider-->>Executor: Final result or terminal error
  Executor-->>Client: Final/error and DONE
Loading

Reviews (2): Last reviewed commit: "fix(canvas): wire agent stream chrome in..." | Re-trigger Greptile

Run-from-block executions emit the same live stream:thinking/stream:tool
events as full runs but registered none of the handlers, so the terminal
never showed thinking or tool chips on that path. The per-run chrome
(batched thinking writes + tool chip lifecycle + settlement on stream done,
block error, and every terminal execution state) is extracted into a shared
createAgentStreamChrome factory consumed by both paths.
@icecrasher321

Copy link
Copy Markdown
Collaborator

@greptile-apps

@icecrasher321

Copy link
Copy Markdown
Collaborator

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 9ef823d. Configure here.

@icecrasher321
icecrasher321 merged commit d24bc7e into staging Jul 24, 2026
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants